Skip to content

feat: GitHub repo-host vertical with live-tests CI#3

Merged
goldenwitch merged 16 commits intomainfrom
feature/ado-ghe-support
May 4, 2026
Merged

feat: GitHub repo-host vertical with live-tests CI#3
goldenwitch merged 16 commits intomainfrom
feature/ado-ghe-support

Conversation

@goldenwitch
Copy link
Copy Markdown
Owner

@goldenwitch goldenwitch commented May 4, 2026

Lands the GitHub repo-host vertical end-to-end: the RepoHostAdapter abstraction, a working github.com adapter, a fully validated live-test fixture pipeline running against a real repo via short-lived GitHub App installation tokens, and a canonical sample exercising the public library surface.

GHE and Azure DevOps adapters compile and have unit tests, but are not wired into routing or PR monitoring. They ship as preview-only scaffolding so the next vertical can plug in without re-doing the abstraction. Tracked in follow-up issues #4, #5, #6.

What's working

RepoHostAdapter abstraction (Phases 4ΓÇô7)

  • RepoHostAdapter trait, RepoHostId, host-aware PrRef/events/ledger keys.
  • New CredentialStore retires AgentSecrets; daemon credentials reworked.
  • Scenario S08 + spec at docs/internals/spec-repo-hosts.md.

github.com adapter ΓÇö fully wired

  • Used by the daemon, the live tests, and samples/pr-reviewer.
  • Credential resolution via env-var ΓåÆ gh CLI provider chain.
  • Validated by dogfooding: the daemon reviewed this PR (commit 96e29c7) and surfaced four real bugs that became the follow-up issues below.

Live test fixtures (devdev-test-env crate)

  • Hand-rolled provisioner: apply / verify / print-env / reset-comments.
  • Manifest at test-env/manifest.json, lockfile at test-env/manifest.lock.json.
  • Fixture repo: goldenwitch/devdev-test-environment (private).

CI: live-tests workflow

  • Four jobs: gate ΓåÆ provision (admin App) ΓåÆ live-tests (consumer App) ΓåÆ cleanup (admin App, gated on provision == success).
  • Tokens minted at job start via actions/create-github-app-token@v1. No long-lived secrets in the repo.
  • Environment split (live-tests-admin / live-tests-consumer) gates blast radius.
  • ADO env-gated via DEVDEV_LIVE_ADO_ENABLED; off until the multi-host follow-ups land.
  • Validation run: Γ£à all four jobs green on run 25293855397.

Credential leak hardening

  • RedactedString (daemon) and Token (test-env) newtypes redact via Debug/Display; expose() is the only path to raw bytes.
  • Audit of 5 surfaces (logs, errors, payload bodies, panic prints, CI echo); all clean.

Agent-spawn consolidation (dogfood-driven)

  • New devdev_cli::agent_command::prepare ΓÇö single canonical entry-point: PATH+PATHEXT resolution ΓåÆ Copilot SEA-bypass rewrite ΓåÆ spawn-ready (program, args). Replaces three scattered call sites.
  • Fixed: devdev send on Windows used to fail with program not found because bare Command::new("copilot") doesn't apply PATHEXT.

samples/pr-reviewer ΓÇö canonical library-surface sample

  • Polls a GitHub repo and has a Copilot ACP agent review each PR. Read-only.
  • Exercises every crate's public API without going through the daemon. Workspace member, so future API breakage fails cargo build --workspace immediately.

What's preview-only (deferred)

Test skips on hosted runners

  • live_workspace_cwd, cargo_builds_hello_world_inside_mount ΓÇö host-bound (FUSE / /home/agent perms), not fixture-related.

goldenwitch added 15 commits May 3, 2026 12:36
Phase 1+2+3 of the ADO/GHE work:

* Rename `GitHubAdapter` trait -> `RepoHostAdapter`, `GitHubError` -> `RepoHostError`.
  The trait now exposes `host_id()` so downstream code can key on the forge
  instance.

* New `host` module with `RepoHostKind` (`GitHub`, `AzureDevOps`) and
  `RepoHostId` (kind + api_base + browse host). Includes a permissive URL
  host classifier (github.com / ghe.* / dev.azure.com / *.visualstudio.com)
  and `from_browse_host` for URL-driven dispatch.

* `GitHubAdapter` (concrete struct, was `LiveGitHubAdapter`) now takes a
  `RepoHostId` and threads `api_base` through every URL formatter, so the
  same impl serves github.com and any GitHub Enterprise Server install
  (https://<ghe>/api/v3). Convenience constructors: `github_com`, `ghe`,
  `from_env`, `new`.

* New `AzureDevOpsAdapter` against ADO REST 7.0 with documented mappings:
  PR `status` -> `PrState`, reviewer vote integers -> `ReviewEvent`, and
  PR Status policy `state` -> `CheckRun.status`/`conclusion`. ADO's
  org/project/repo triple is encoded as owner="<org>/<project>",
  repo="<repo>". Auth via HTTP Basic with empty user + PAT.

* Renamed `MockGitHubAdapter` -> `MockAdapter`; default host is github.com
  but `with_host` lets tests simulate any forge.

* Daemon dispatch + tasks + tests rewired to the trait rename.
  `select_github_adapter` now reads `DEVDEV_REPO_HOST_ADAPTER`, with
  `DEVDEV_GITHUB_ADAPTER` retained as a legacy alias.

Workspace `cargo build` + `cargo clippy --workspace --all-targets -- -D
warnings` + `cargo test --workspace --tests` are green. The pre-existing
WinFSP-related `live_workspace_cwd` test still fails with
STATUS_DLL_NOT_FOUND on this machine (unchanged from main).

Follow-ups on this branch: credential snapshot abstraction, daemon host
registry, multi-host PR ref parsing, preferences `[[repo]]` entries,
scenario S08.
Snapshot-once credential lifecycle for multi-host auth.

- credentials.rs: CredentialProvider trait + EnvVarProvider,
  GhCliProvider, AzCliProvider built-ins; frozen CredentialStore
  keyed by RepoHostId; RedactedString wrapper that never leaks via
  Debug/Display.
- dispatch.rs: replace Arc<Mutex<AgentSecrets>> with Arc<CredentialStore>.
- mcp/provider.rs: DaemonToolProvider takes Arc<CredentialStore>;
  ask() resolves github.com from the snapshot for token-bearing kinds.
- daemon_cli::run_up: build snapshot from EnvVar(GH_TOKEN) -> GhCli
  chain, log captured source.
- secrets.rs deleted; e2e_pr_shepherding migrated.

Tests (11 new, all passing):
  - redacted_string_does_not_leak_via_debug_or_display
  - snapshot_records_token_for_each_host
  - snapshot_falls_through_on_none_within_chain
  - snapshot_first_provider_wins_when_both_have_token
  - store_is_immutable_after_snapshot_env_mutation (lifecycle gate)
  - env_var_provider_{unset,empty,records_source}
  - with_entry_round_trips, empty_store_returns_none_and_is_clonable
  - expires_at_hint_is_one_hour_after_sample

cargo clippy --workspace --all-targets -- -D warnings clean.
All workspace tests pass except pre-existing WinFSP
live_workspace_cwd failure (unchanged from main).
PrRef carries a RepoHostId so URLs to GitHub.com, GHE, dev.azure.com, and *.visualstudio.com all parse into a single host-tagged value. RepoWatchTask, MonitorPrTask, and DaemonEvent now track host_id end-to-end so identical (owner, repo, number) triples on different hosts cannot collide in dispatch maps, ledger keys, or the event bus. dispatch.rs threads host_id through repo/watch, repo/unwatch, task/add, and ensure_monitor_pr_task; the params.host field defaults to github.com for back-compat. Phase 5 will replace the default with registry-driven routing.
RepoHostRegistry maps RepoHostId to RepoHostAdapter so dispatch layers route to the correct API surface for github.com, GHE, and dev.azure.com hosts. for_url() classifies a browser-shaped URL or bare host via RepoHostId::from_browse_host and looks up the registered adapter. DispatchContext exposes a host_registry alongside the legacy github default; adapter_for(&host_id) prefers a registered adapter and falls back to github so single-host smoke flows keep working. AskRequest now carries an optional host field that selects which credential entry the response surfaces; unknown hosts are rejected hard. 10 new tests (8 registry + 2 ask routing).
docs/internals/spec-repo-hosts.md captures the multi-host architecture: RepoHostId/Adapter/Registry/CredentialStore seams, identity invariants, URL parsing matrix, the wire surface for repo/watch and devdev_ask host fields, and a test-landmark map. Scenario S08 exercises the full IPC surface end-to-end: same (owner, repo) on github.com vs ghe.acme.io must produce distinct task ids, idempotent re-watch returns the same id, unknown hosts hit -32602, and host-keyed unwatch never collapses across hosts.
New crate `devdev-test-env` (publish=false) reads a JSON manifest at `test-env/manifest.json` and reconciles GitHub.com + Azure DevOps fixtures via hand-rolled REST. Server-assigned ids land in a committed `manifest.lock.json`.

Subcommands: apply (idempotent reconcile), verify (drift check), reset-comments (per-test cleanup; non-admin always swept, admin swept only if body carries the [devdev-live-test...] tag), print-env (emits DEVDEV_GH_PR_URL + ADO equivalents for the test runner), destroy (intentionally not implemented in first cut).

CI: .github/workflows/live-tests.yml runs a three-job pipeline (provision -> live-tests -> cleanup). Admin tokens live in the live-tests-admin GitHub Environment and are visible only to provision and cleanup; consumer tokens live in live-tests-consumer and are visible only to the live-tests job. Triggered by workflow_dispatch, nightly cron, or PRs labeled `live-tests`. CODEOWNERS protects the workflow, manifest, and crate.

Docs: docs/internals/live-test-fixtures.md (operational story, bootstrap, principal model, cost) and docs/internals/ghe-gap.md (deliberate GHE gap, what we still rely on, sponsorship invitation).

Claim FIXTURE-MANIFEST-INTEGRITY covers the deterministic side (manifest validation + reset decision logic, 14 unit tests, validate.ps1 PASS). The fixture-state-matches-manifest side runs in CI only because it requires admin tokens.
Three new `#[ignore]`-gated live tests in devdev-cli, each with a separate env flag so CI can opt them in independently:

* live_host_probe.rs: GitHubAdapter and AzureDevOpsAdapter round-trip the canonical fixture PR (get_pr + list_open_prs); asserts host_id stamp matches PrRef classification. Gate: DEVDEV_LIVE_HOSTS=1.

* live_credential_chain.rs: GhCliProvider and AzCliProvider produce non-empty tokens from real signed-in CLIs; asserts TokenSource is GhCli/AzCli respectively. Gate: DEVDEV_LIVE_CRED_GH=1 / DEVDEV_LIVE_CRED_AZ=1.

* live_ado_pr.rs: read path always (DEVDEV_LIVE_HOSTS=1); write path posts a tagged comment ([devdev-live-test:live_ado_pr:<nonce>]) and verifies it lands in the next list_pr_comments. Write path gated by DEVDEV_LIVE_WRITE=1; cleanup happens via devdev-test-env reset-comments. #[serial] to avoid nonce collisions.

Wired into .github/workflows/live-tests.yml: the live-tests job now sets DEVDEV_LIVE_HOSTS=1 and DEVDEV_LIVE_CRED_GH=1 by default; DEVDEV_LIVE_CRED_AZ stays empty until the workflow seeds az login. Five new claims in claims.toml, surfaced in VALIDATION.md. README.md gains a pointer to the live-test fixture docs.
- Wrap GithubClient.token + AdoClient.auth_header in a Token newtype that redacts via Debug/Display; expose() is the only path to the raw value.

- Rewrite .github/workflows/live-tests.yml to mint installation tokens via actions/create-github-app-token@v1 from environment-scoped vars + secrets. ADO portions are env-gated (DEVDEV_LIVE_ADO_ENABLED) until federated creds land.

- Add scripts/devdev-secrets.ps1 (vault helper), scripts/verify-gh-apps.ps1, scripts/verify-gh-e2e.ps1 (raw-REST E2E), scripts/seed-ci-secrets.ps1 (idempotent CI bootstrap from vault).

- Rename canonical fixture to goldenwitch/devdev-test-environment; add test-env/manifest.lock.json captured from live apply.
Discovered while dogfooding the new keyring chain. The daemon's startup capture step was correctly pulling the gh CLI token into the CredentialStore, but select_github_adapter was still calling GitHubAdapter::from_env(), which only consults GH_TOKEN. Result: 'credential captured (source: GhCli)' followed immediately by 'falling back to mock adapter'. Thread the CredentialStore into the selector so the live adapter uses whichever provider won (env var or gh CLI).
The CLI used to scatter agent-launch logic across three places: acp_backend.rs called rewrite_copilot_invocation inline, realpath_shim.rs owned the Windows-only Copilot SEA bypass, and the live_daemon_fs_write test rolled its own PATHEXT search. Bare 'copilot' on Windows then went straight to Command::new (which doesn't apply PATHEXT for extensionless names), failing with 'program not found' even though the index.js was sitting right next to the .cmd shim and the rewrite would have worked if we'd resolved the launcher first.

Introduce devdev_cli::agent_command::prepare(program, args) as the single canonical entry-point: PATH+PATHEXT resolution -> Copilot SEA-bypass rewrite -> spawn-ready (program, args). acp_backend now calls prepare() once; the live test reuses resolve_on_path. Verified end-to-end: 'devdev send pong' now spawns Copilot, gets a reply, and emits exactly one INFO line announcing the chosen launch path.
cleanup ran with 'needs: live-tests' + 'if: always()' but unconditionally downloaded the manifest-lock artifact. If gate skipped the workflow or provision failed before upload, live-tests was skipped and cleanup still ran, then failed on the missing artifact -- turning a clean skip/failure into a confusing secondary cleanup failure. Make cleanup depend on provision and only run when provision succeeded (it's the only job that uploads manifest-lock and the only thing cleanup needs to undo).
Polls a GitHub repository for open PRs and asks a Copilot ACP agent to review each one as it appears or updates. Read-only; never posts to the PR.

The point of this sample is to exercise the *library* surface of every DevDev crate without going through the daemon or its IPC. Each crate contributes exactly one thing: devdev-acp gives us AcpClient::connect_process, devdev-cli gives us agent_command::prepare (the canonical resolver+rewriter), devdev-daemon gives us the CredentialStore + provider chain, devdev-integrations gives us GitHubAdapter and pr_state_hash. If anything in this sample ends up doing the daemon's work by hand, that's a signal a daemon-internal helper should be promoted to a library API.

Validated end-to-end: 'cargo run -p pr-reviewer -- goldenwitch/devdev --once' reviewed PR #3, found 3 substantive issues, and exited cleanly. Builds clean as a workspace member, so any future API breakage in the public surface fails 'cargo build --workspace' immediately.
# Conflicts:
#	.github/workflows/live-tests.yml
@goldenwitch goldenwitch merged commit 68f6526 into main May 4, 2026
4 of 5 checks passed
@goldenwitch goldenwitch deleted the feature/ado-ghe-support branch May 4, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant